Correlations Among Maximum Likelihood and Weighted/Unweighted Least Squares Estimators in Factor Analysis

Least squares and maximum likelihood

Least squares methods in maximum likelihood problems

It is well known that the GaussNewton algorithm for solving nonlinear least squares problems is a special case of the scoring algorithm for maximizing log likelihoods. What has received less attention is that the computation of the current correction in the scoring algorithm in both its line search and trust region forms can be cast as a linear least squares problem. This is an important observ...

Origins of the limited information maximum likelihood and two-stage least squares estimators

Theil, Basmann, and Sargan are often credited with the development of the two-stage least squares (TSLS) estimator of the coefficients of one structural equation in a simultaneous equations model. However, Anderson and Rubin had earlier derived the asymptotic distribution of the limited information maximum likelihood (LIML) estimator by finding the asymptotic distribution of what is essentially...

Estimation by Least Squares and by Maximum Likelihood

Although statements to the contrary are often made, application of the principle of least squares is not limited to situations in which p is normally distributed. The GaussMarkov theorem is to the effect that, among unbiased estimates which are linear functions of the observations, those yielded by least squares have minimum variance, and the independence of this property from any assumption re...

Mixing Least - Squares Estimators

We propose a procedure to handle the problem of Gaussian regression when the variance is unknown. We mix least-squares estimators from various models according to a procedure inspired by that of Leung and Barron [17]. We show that in some cases the resulting estimator is a simple shrinkage estimator. We then apply this procedure in various statistical settings such as linear regression or adapt...